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ABSTRACT 



The American College Testing Program (ACT) routinely 
evaluates the fit of its tests to the curriculum of high schools nationwide 
to ensure that what is being assessed reflects what is being taught as 
necessary for college success. This report describes the steps ACT has taken 
to ensure that the content foiindations of the ACT Assessment Mathematics Test 
are solid. Evidence is provided from: (1) an account of how the content and 

skills measured by the ACT Assessment Mathematics Test are determined and how 
they are kept up to date; (2) survey results demonstrating that what the 
Mathematics Test measures is consistent with what is being taught in high 
schools that is necessary for college success; (3) reviews of state 
curriculum frameworks and national curriculum and performance standards 
supporting the validity of the test at the high school level; (4) a summary 
of ACT'S review of current mathematics textbooks nationwide; (5) 
documentation of the role U.S. mathematics teachers play in developing the 
ACT mathematics test; (6) results of predictive validity and placement 
studies that show the mathematics test is effective in measuring the 
mathematical skills important for college success. (Author/PVD) 
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ACT and the ACT Assessment 



ACT was founded in 1959 as a not- 
for-profit organization and has grown to 
become one of the leading providers of 
educational assessment and research 
services in the world. With its focus on 
providing information for life's transi- 
tions, ACT provides a wide range of 
services and programs in areas such as 
educational assessment, career planning, 
student financial aid, educational data 
management, and educational and 
psychometric research. 

act's flagship program, the ACT 
Assessment Program, provides a wide 
range of assessment information and 
services to help students make a 
successful transition from high school to 
postsecondary education. Over 1.5 
million tests are administered each year 
to college-bound students. The ACT 
Assessment tests include questions that 
measure problem-solving and critical- 
reasoning skills across the major 
curricular areas of English, mathematics, 
reading, and science reasoning. Materials 
taken from the humanities (arts, music, 
philosophy, theater, architecture), prose 
fiction (short stories, novels), social stud- 
ies (history, political science, economics, 
anthropology, psychology, sociology), 
and natural sciences (biology, chemistry, 
physics. Earth/space science) serve as 
the contexts for the measurement of 
these skills. These materials ask students 
to solve problems, draw inferences, 
interpret information, evaluate ideas, 
and compare concepts — that is, to apply 
the knowledge and skills they have 
learned in major curriculum areas. 



The ACT Assessment Program also 
includes a needs assessment and an 
interest inventory. The needs assessment 
allows students to identify academic 
areas in which they need some help, a 
lot of help, or no help. This provides 
both high school and college faculty with 
necessary information to help students 
be academically successful. The interest 
inventory permits students to identify 
areas of vocational interest so that the 
students can compare their interests, 
academic performance, and career plans, 
to make sure they are complementary. 

Students can also self-report 
important information about themselves. 
This information can be used by colleges 
and other institutions to better 
understand and advise incoming 
students. 

Information from the ACT 
Assessment is provided to students and 
high schools, as well as to colleges 
designated by the student. Based on this 
information and other information 
provided by the high school, such as 
high school grade point average and a 
course transcript, decisions can be made 
by colleges about admissions and course 
placement. Similarly, ACT Assessment 
information can be used by colleges and 
other institutions to make scholarship 
awards. The ACT Assessment Program 
offers a well-rounded picture of the 
college applicant, thus enhancing the 
student's chances for success in making 
the transition from high school to 
college. 
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Introduction 



The ACT Assessment Program was established by E. F. Lindquist, who 
believed that the test should measure what students can do with what they 
have learned throughout their schooling. Further, this measurement 
needed to be grounded in the kinds of courses students typically take in 
high school that are important for college success. Dr. Lindquist's 
philosophy has guided the specifications for the ACT Assessment 
educational achievement tests, both when they were created and today. 

The ACT Assessment Mathematics Test is designed to assess the 
mathematical skills that students have typically acquired in courses taken 
up to the beginning of grade 12 and that are relevant to success in college. 
The test is one of four given as part of the ACT Assessment Program; all 
four tests have been designed to reflect today's curriculum. The ACT 
Assessment is the only curriculum-based college admissions and placement 
test available in the United States today that focuses on what is important 
for successful high school -to -college transitions. 

ACT routinely evaluates the fit of its tests to the curriculum to ensure that 
what is being assessed reflects what is being taught in classrooms 
nationwide that is necessary for college success. This report describes the 
steps ACT has taken to ensure that the content foundations of the ACT 
Assessment Mathematics Test are solid. Evidence is provided from these 
sources: 

• an account of how the content and skills measured by the ACT 
Assessment Mathematics Test are determined and how they are kept 
up to date, 

• survey results demonstrating that what the ACT Assessment 
Mathematics Test measures is consistent with what is being taught in 
high schools today that is necessary for college success, 

• reviews of state curriculum frameworks and national curriculum and 
performance standards supporting the validity of the ACT Assessment 
Mathematics Test at the high school level, 

• a summary of ACT's review of current mathematics textbooks 
nationwide, 

• documentation of the role that U.S. mathematics teachers play in 
developing the ACT Assessment Mathematics Test, and 

• results of predictive validity and placement studies that show the ACT 
Assessment Mathematics Test is effective in measuring mathematical 
skills important for success in college. 
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Test Specifications Development 



Test specifications are the blueprints that are used to construct a test. They 
are composed of two basic parts: (1) a description of the content and skills 
to be measured by the test and (2) a description of the statistical 
characteristics of the test questions. This report focuses on the content and 
skill specifications. 

The original content specifications for the ACT Assessment Mathematics 
Test, like those of the other three ACT Assessment tests, were determined 
through a detailed analysis and review of the curriculum. To maintain the 
accuracy of the specifications and to monitor the secondary-postsecondary 
curriculum balance, ACT routinely repeats a similar analysis and review 
process. Test specifications are then refined or modified as appropriate. 

ACT gathers information in the following ways. 

• High school teachers and college faculty are consulted to determine the 
knowledge and skills taught up to grade 12 that are required for 
successful performance in mathematics in college. This information, in 
survey form, provides lists of currently taught topics upon which to 
base the test specifications. 

» The curriculum frameworks for grades 7 through 12 are obtained for all 
states in the United States that have published such frameworks. This 
information is used to identify and add any areas of the curriculum 
that were not covered in the survey process. These frameworks are 
reviewed and combined with the results of the surveys. 

• Textbooks on state-approved lists for courses in grades 7 through 12 
are reviewed. ACT uses the information from the textbooks to round 
out the topic list of what is being taught in high school mathematics 
courses. The textbook review also aids in understanding how students 
are being prepared for successful college mathematics performance. 

These three sources of information are then analyzed to define a scope and 
sequence for mathematics learning. 

ACT also tries to project how the curriculum may evolve. By reviewing 
materials such as the National Council of Teachers of Mathematics 
(NCTM) Standards, the National Assessment of Educational Progress 
(NAEP) mathematics framework, and other national efforts related to Goal 
5 ("By the year 2000, United States students will be the first in the world in 
mathematics and science achievement") of the federal Goals 2000 initiative, 
ACT identifies emerging curricular trends that are likely to have an impact 
on the ACT Assessment. 
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The information ACT collects on the current and emerging curricular 
trends is then shared with panels of mathematics experts, including 
teachers, curriculum specialists, state department of education specialists, 
and college faculty in mathematics and education. Their reviews aid ACT 
in confirming the appropriateness of refinements to the specifications. 

The results of all these analyses and reviews are carefully evaluated, 
resulting in the test specifications for the ACT Assessment Mathematics 
Test. The primary consideration in building the specifications is to 
maintain a delicate balance between what is being taught in the high 
schools and what are necessary skills for success in college-entry course 
work. 

The ACT Assessment Mathematics Test 



Table 1 provides a summary of the content specifications for the ACT 
Assessment Mathematics Test. The test presents multiple-choice questions 
that require students to use their reasoning skills to solve practical 
problems in mathematics. The problems assume knowledge of basic 
formulas and computational skills but do not require complex formulas 
nor extensive computations. The material covered on the test emphasizes 
the major content areas that are prerequisite to successful performance in 
entry-level courses in college mathematics. 

Table 1 

ACT Assessment Mathematics Test Content Specifications 



Content Area 


Number and Percentage 
of Questions 


Pre-Algebra 


14 (23%) 


Elementary Algebra 


10 (17%) 


Intermediate Algebra 


9 (15%) 


Coordinate Geometry 


9 (15%) 


Plane Geometry 


14 (23%) 


Trigonometry 


4 ( 7%) 


Total Questions 


60 



• Pre-Algebra. Questions in this content area are based on basic operations 
using whole numbers, decimals, fractions, and integers; place value; 
square roots and approximations; the concept of exponents; scientific 
notation; factors; ratio, proportion, and percent; linear equations in one 
variable; absolute value and ordering numbers by value; elementary 
counting techniques and simple probability; data collection. 
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representation, and interpretation; and understanding simple 
descriptive statistics. 

• Elementary Algebra. Questions in this content area are based on 
properties of exponents and square roots, evaluation of algebraic 
expressions through substitution, using variables to express functional 
relationships, understanding algebraic operations, and solving 
quadratic equations by factoring. 

® Intermediate Algebra. Questions in this content area are based on an 
understanding of the quadratic formula, rational and radical 
expressions, absolute value equations and inequalities, sequences and 
patterns, systems of equations, quadratic inequalities, functions, 
modeling, matrices, roots of polynomials, and complex numbers. 

o Coordinate Geometry. Questions in this content area are based on 
graphing and the relations between equations and graphs, including 
points, lines, polynomials, circles, and other curves; graphing 
inequalities; slope; parallel and perpendicular lines; distance; 
midpoints; and conics. 

o Plane Geometry. Questions in this content area are based on the 
properties and relations of plane figures, including angles and relations 
among perpendicular and parallel lines; properties of circles, triangles, 
rectangles, parallelograms, and trapezoids; transformations; the concept 
of proof and proof techniques; volume; and applications of geometry to ^ 
three dimensions. 

• Trigonometry. Questions in this content area are based on 
understanding trigonometric relations in right triangles; values and 
properties of trigonometric functions; graphing trigonometric functions; 
modeling using trigonometric functions; use of trigonometric identities; 
and solving trigonometric equations. 

Although the current test specifications do not specifically list "probability 
and statistics" as a separate category, questions in this area are included 
on the ACT Assessment Mathematics Test within other content categories 
such as Pre-Algebra. ACT is including more of these questions as the topic 
becomes more common in mathematics classrooms and increases in 
importance for success in college. 

The questions included in the ACT Assessment Mathematics Test cover 
four cognitive levels: knowledge and skills, direct application, 
understanding concepts, and integrating conceptual understanding. 

Numbers and percentages of each type of question in the test are 
presented in Table 2. 
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• Knowledge and Skills. These questions require the student to use one or 
more facts, definitions, formulas, or procedures to solve straight- 
forward problems that are presented in purely mathematical terms. 

® Direct Application. These questions require the student to use one or 
more facts, definitions, formulas, or procedures to solve straight- 
forward problems set in real-world situations. 

• Understanding Concepts. These questions test the student's depth of 
understanding of major concepts by requiring reasoning from a concept 
to reach an inference or a conclusion. 

• Integrating Conceptual Understanding. These questions test the student's 
ability to achieve an integrated understanding of two or more major 
concepts to solve nonroutine problems. 

Table 2 

ACT Assessment Mathematics Test Cognitive Level Specifications 



Cognitive Level 


Number and Percentage 
of Questions 


Knowledge and Skills 


32 (53%) 


Direct Application 


16 (26%) 


Understanding Concepts/ 


12 (20%) 


Integrating Conceptual Understanding 


Total Questions 


60 



Beginning in fall 1996, students were permitted to use calculators on the 
ACT Assessment Mathematics Test. No changes in the test specifications 
were required to permit the addition of calculator use. ACT's research 
studies showed that the scale scores reported on the ACT Assessment 
Mathematics Test continue to have the same meaning as they had before 
calculators were permitted. 

Updating Test Specifications 



act's most recent national mathematics curriculum study to define and 
refine the test specifications was conducted in 1994-95. At that time, ACT 
reviewed the content specifications of all four academic tests in the ACT 
Assessment and their fit to the secondary-postsecondary curriculum 
continuum. The results of the reviews in mathematics are described in the 
following sections. 
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Curriculum Survey 

In the spring of 1995, ACT conducted a mathematics curriculum survey 
targeted at 4,800 high school teachers, curriculum specialists, and 
department heads, as well as college faculty. Half of the 4,800 mathematics 
educators were from the secondary level and half from the postsecondary 
level. These groups were each further subdivided into equal groups — half 
responding to questions about various elements of probability and 
statistics, geometry, and trigonometry and half responding to questions 
about various elements of algebra and calculus. (The subdividing was 
required to make it possible to complete the survey in a reasonable amount 
of time.) All groups were asked about general mathematics skills. High 
school respondents were asked to rate the importance of the survey topics 
in high school mathematics courses. College and university respondents 
were asked to rate the importance of the topics as prerequisites for success 
in entry-level and advanced-level college mathematics courses. The 
response rate for each secondary group was 27%. The response rate for 
each postsecondary group was 16%. 

Weighted numbers of respondents and their respective regions appear in 
Table 3. 



Table 3 

Weighted Number of Survey Responses by Geographic Region 



Region 


Weighted Sample 


Secondary 


Post- 

secondary 


Pacific/Mountain 


26 


13 


North Central 


10 


10 


South Central 


19 


23 


Southeast 


10 


15 


Northeast 


19 


30 


Great Lakes 


26 


29 



All of the respondents were asked to consider the relative importance of 
mathematics content and skills that a typical student bound for a 4-year 
college needs to know. Secondary respondents were also asked the grade 
level at which the content or skill is first introduced to that student in the 
respondent's school or district. The surveys included topics in general 
mathematics skills, as well as the content areas of algebra, geometry, 
trigonometry, probability and statistics, and calculus. Space was provided 
for respondents to list other topics as desired. These added topics were 
subsequently reviewed by ACT staff and considered in the final 
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specifications review. Results of the responses were reported on a 5-point 
scale, where 1 = no importance, 3 = medium importance, and 5 = great 
importance. The responses from all secondary and all postsecondary 
respondents were combined for evaluation. The appendix contains a list of 
all survey topics and the mean ratings for each by respondent group. 

The data described below are based on averages of the ratings of each 
topic by the respondents, with higher averages reflecting greater 
importance. Average ratings at or above 3 (medium importance) were 
tabulated. 

General Mathematics Skills 

The surveys listed 21 general mathematical skills. Of these 21 topics, 
college faculty reported that all but 6 of them (a total of 71%) were at least 
moderately important for entry-level freshmen to have mastered. High 
school teachers reported that all 21 topics were at least moderately 
important for college-bound students to know. Clearly, the kinds of 
general skills students acquire in their mathematics classes are important at 
both levels, high school and college. 

The ACT Assessment reflects this emphasis, in that 14 of the topics are 
directly assessed on the Mathematics Test. Of the 7 topics ACT did not 
include on the Mathematics Test, 5 of them were rated below 3 for entry- 
level freshmen, indicating a high level of agreement between the test 
specifications and entry-level requirements in the area of mathematics 
skills. Those skills excluded from the ACT Assessment Mathematics Test 
specifications and rated lower than 3 for entry-level freshmen were; 
constructing an inductive proof, constructing a deductive proof, 
formulating new patterns or structures, performing statistical operations 
with a calculator, and graphing and exploring the nature of equations 
using a graphing calculator. Skills rated above 3 for entry-level freshmen 
but not directly assessed on the ACT Assessment Mathematics Test are 
communicating mathematically (orally) and communicating mathematically 
(in writing). 

College faculty found 19 of the 21 topics, or 90%, to be important for 
advanced-level freshmen (rated at 3 or higher). The two topics rated lower 
were constructing an inductive proof and performing statistical operations 
with a calculator. 

Algebra 

A total of 46 algebra topics were listed in the survey. Of these topics, 
college faculty told ACT that 27 topics (59%) were at least moderately 
important. On the other hand, high school teachers reported that all but 
one of the topics (finding common factors and common multiples) were 
important from their perspective. 
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Of the 46 algebra topics included in the survey, 100% are included in the 
ACT Assessment Mathematics Test specifications. While the topic list 
encompasses more than what may be needed for entry-level freshmen, all 
but 4 of the topics on the algebra list were included as needed for 
advanced-level freshmen. These results indicate that the ACT Assessment 
Mathematics Test has a content basis in algebra for distinguishing between 
entry-level and advanced-level freshmen. This information can be useful in 
helping colleges and universities place incoming freshmen more effectively. 
The Mathematics Test produces 3 subscores: Pre- Algebra /Elementary 
Algebra, Intermediate Algebra /Coordinate Geometry, and Plane 
Geometry/Trigonometry. Algebra is an important component of two of the 
subscores, both of which can serve as useful bases for making placement 
decisions. 

Geometry 

There were 36 topics on the geometry list. College faculty told ACT that 16 
of them (44%) were at least moderately important for entry-level students. 
High school teachers, on the other hand, told ACT that 35 of the 36 were 
important, rating only "constructing and using truth tables" lower than 3. 
This high school -to -college discrepancy in emphasis was detected earlier 
in a 1985 survey and continues today. However, as with the algebra topics, 
the college faculty told ACT that knowledge of geometry was important at 
the advanced-placement level. Of the 36 topics, 30 (83%) were rated as 
being at least moderately important for advanced-level freshmen. 

The ACT Assessment Mathematics Test covers all of the topics college 
faculty rated highly and all but 4 of the topics high school teachers rated 
highly. 

Because some of the geometry skills are valued for advanced-level 
freshmen and not for entry-level freshmen, the ACT Assessment 
Mathematics Test has a content basis for distinguishing between entry- 
level and advanced-level freshmen in geometry, as was seen in algebra. 
Geometry, as it appears in the subscores of Intermediate Algebra/ 
Coordinate Geometry and Plane Geometry/Trigonometry, can also be used 
as a basis for making placement decisions in mathematics at the college 
level. 

Probability and Statistics (P&S) 

Of the 9 survey topics listed for P&S, college faculty said none of them 
were important enough to be rated higher than 3. High school teachers 
said all of them were as important as 3 or higher. This discrepancy may be 
an indication of the increasing emphasis being placed on P&S at the high 
school level, resulting from efforts such as the NCTM Standards. Colleges, 
however, do not yet find knowledge of the area to be necessary for success 
in entry-level or advanced-level freshmen mathematics classes. 
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The ACT Assessment Mathematics Test encompasses all but 2 of the topics 
rated 3 or higher by respondents, excluding "testing a null hypothesis" 
and "deterrnining confidence intervals." The 6 remaining topics are 
included in the Pre-Algebra part of the test specifications. ACT's 
experience has been that these types of questions are currently too difficult 
to include on the ACT Assessment Mathematics Test. At some future time, 
when students are more experienced in P&S, these topics may become less 
difficult, and may then be included on the ACT Assessment Mathematics 
Test. 

Trigonometry 

The ACT surveys listed 6 topics under trigonometry. High school teachers 
told ACT that all 6 topics were important. Although college faculty said 
that none of the topics were important for entry-level freshmen, the faculty 
did find all 6 topics important for advanced-level freshmen. 

The ACT Assessment Mathematics Test encompasses all 6 of the 
trigonometry topics. Because of the differential valuing of trigonometry for 
entry-level as compared to advanced-level freshmen, the Mathematics Test 
score and its Plane Geometry/Trigonometry subscore can be helpful in 
placing entering freshmen. 

Calculus 

Five calculus topics were listed in the surveys. High school teachers 
reported that all of the topics were important from their perspective. 
However, college faculty found none of them important for entry-level 
freshmen. Four of the topics — finding derivatives; determining maxima, 
minima, and points of inflection for functions; understanding continuity; 
and finding the limit of an expression — were found important for 
advanced-level freshmen. 

The ACT Assessment Mathematics Test does not require students to 
demonstrate specific knowledge or skill in calculus but, in some instances, 
students may use calculus concepts to answer a question, such as finding 
the minimum value of a parabola. In general, ACT has found that calculus 
questions are too difficult for the general ACT-tested population, often 
because students have not taken calculus by the 12th grade. ACT therefore 
has not included calculus queshons as part of the test specifications. 

Table 4 lists the median ratings of the mathematics topics by level. Median 
ratings are the midpoints of the ratings, above and below which half of the 
ratings fall. For algebra at the secondary level, the respondents found, on 
average, that algebra was somewhat important (4 = somewhat important 
on the l-to-5 scale), with a median rating of 3.9. As may be determined 
from reviewing the table, the median ratings of the topics by secondary 
teachers were higher for all content areas than were any of the ratings by 
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college faculty for entry-level students. For advanced-level freshmen, the 
median ratings were approximately as high as those for secondary 
students. One notable exception is probability and statistics, which was 
rated far lower for freshmen in either group than for secondary students. 

Table 4 

Median Survey Ratings for Mathematics Content Areas 
(based on a 1 -5 scale) 



Content Area 


Academic Level 


Secondary 

School 


Entry-level 

College 

Freshmen 


Advanced-level 

College 

Freshmen 


Mathematics Skills 


4.0 


3.4 


3.9 


Algebra 


3.9 


3.2 


4.0 


Geometry 


3.9 


2.8 


3.9 


Probability & Statistics 


3.4 


1.8 


1.8 


Trigonometry 


3.8 


2.1 


3.7 


Calculus 


4.2 


1.4 


2.9 



Additional Sources of Content Validity 



This section describes the kinds of comparisons ACT has performed with 
the state curriculum frameworks, the NCTM Standards, and the NAEP 
mathematics framework. As an indicator of the potential trend of the 
curriculum, the NCTM Standards represent future directions the curriculum 
may take. The NAEP framework forms the basis for a national assessment 
often closely tied to state assessment programs and therefore closely 
related to state curricula. The state frameworks, in turn, depict the 
immediate future of the curriculum. This section also describes how 
teachers are involved in the development of the ACT Assessment 
Mathematics Test. 

State Curriculum Frameworks 

In 1993, ACT contacted all 50 states for copies of their most recent 
mathematics curriculum frameworks. ACT staff read and summarized 
frameworks from the responding 39 states. The summaries were used to 
obtain an overview of the mathematics curriculum nationally. ACT found 
that many states are aligning their curriculum frameworks with the NCTM 
Standards. Problem solving, for example, is the focus of many states' 
recommendations. Other NCTM Standards topics, including valuing 
mathematics, becoming a confident problem solver, mathematical 
communication, mathematical reasoning, and making mathematical 
connections are often emphasized also. Many states recommend that 
calculators and other technology be used extensively in educating students 



in mathematics. Many states emphasize conceptual understanding and 
higher-order thinking. And many states are examining the potential of 
performance assessment at the classroom, school, district, and state levels 
and exploring the problems and benefits associated with it. 

ACT found that the topic of probability and statistics is growing in 
importance within the curriculum, as are the related concepts of presenting 
data, reading charts, and interpreting graphs. Although discrete 
mathematics topics are also growing in importance, other topics, such as 
the use of logarithms as a calculation aid, have virtually disappeared from 
the curriculum, and factoring quadratic expressions, though seen as 
important for understanding relations, is receiving diminishing attention in 
the curriculum. 

When the frameworks are compared to the survey results described in the 
previous section, there is a gap between what is actually taught and what 
is described in the curriculum frameworks. The curriculum frameworks 
tend to provide information about where classroom practice is headed, as 
opposed to what is happening today. For example, neither the high school 
nor the college teachers responding to ACT's survey rated as important the 
affective aspects of mathematics, including "becoming a confident problem 
solver" and "valuing mathematics." Neither did "making mathematical 
connections" emerge from the surveys as an important topic. The 
curriculum review did, however, reinforce the need for real-world contexts 
for mathematical problem solving, conceptual understanding, higher-order 
thinking, and the use of calculators, as key areas of emphasis. As the 
curriculum frameworks become more integrated into practice, those gaps 
will disappear. 

NAEP Frameworks 

The National Assessment of Educational Progress (NAEP) provides 
information about student proficiency in mathematics nationally. Based on 
many of the same principles as the NCTM Standards and many state 
curriculum frameworks, this assessment is being used by states as an 
essential part of their regular testing. Thus, its underlying curriculum 
frameworks both guide and reflect state curriculum frameworks. The 
NAEP frameworks also illustrate curriculum found to be important for 
students nationwide. It was therefore important for ACT to consider the 
NAEP mathematics framework in developing the ACT Assessment 
Mathematics Test. This section of this report compares the NAEP 
framework for mathematics with the ACT Assessment Mathematics Test 
specifications. 

The 1994 NAEP in Mathematics was based on a model consisting of five 
major content strands: number sense, properties, and operations; 
measurement; geometry and spatial sense; data analysis, statistics, and 
probability; and algebra and functions. The ACT Assessment Mathematics 



Test encompasses the first two strands in its Pre-Algebra category. The 
NAEP in Mathematics weights these strands more heavily than does the 
ACT Assessment Mathematics Test, however. The NAEP's "algebra and 
functions" are covered in the Mathematics Test areas of Elementary 
Algebra, Intermediate Algebra, and Trigonometry. The Mathematics Test is 
more heavily weighted in these areas than is the NAEP. The NAEP's 
"geometry and spatial sense" and "measurement" are covered in the ACT 
Assessment Mathematics Test in the categories of Coordinate Geometry 
and Geometry, and the weighting of the categories is approximately equal 
across the two assessments. 

The NAEP in Mathematics subdivides each content strand into topics and 
subtopics. There is a good match between these topics and subtopics and 
those covered by the ACT Assessment Mathematics Test, despite such 
differences as the use of manipulatives and student-constructed responses 
on the NAEP in Mathematics but not on the ACT Assessment Mathematics 
Test. 

The NAEP in Mathematics assesses the content strands across three ability 
levels: conceptual understanding, procedural knowledge, and problem 
solving. The three ability levels are viewed within a broader context of 
reasoning, communication, and connections, and are related to an 
overarching model of mathematical power. Unfortunately, because their 
categories are differently defined, the three ability levels of the NAEP and 
the three cognitive skill levels of the ACT Assessment Mathematics Test 
cannot be directly compared. Their goal is a common one, however: to 
determine students' ability to reason mathematically at a variety of 
cognitive levels. 

The NAEP in Mathematics contains "families" of tasks/questions to probe 
the vertical nature of students' understanding, from giving a definition, to 
applying the defined concept in a familiar setting, to using the concept in 
solving a new problems that might require considerable insight, to 
generalizing the knowledge about the concept to a new level of 
understanding. Another type of "family" might measure horizontal 
knowledge across several content strands. Like the NAEP in Mathematics, 
the ACT Assessment Mathematics Test is designed to measure both the 
breadth and depth of student knowledge in mathematics, often in 
integrated ways. 

The NAEP in Mathematics allows students to use calculators on all phases 
of the assessment that do not measure basic fact, operations, or estimation 
abilities. The calculators are provided by NAEP, and students are given 
time to practice with them. ACT also permits the use of calculators on the 
ACT Assessment Mathematics Test, agreeing with NAEP that calculator 
use is appropriate given the use of calculators that students encounter both 
in and beyond the mathematics classroom. ACT permits students to bring 



the calculator with which they are most familiar, within certain limits 
dictated by test security concerns. 

NCTM Standards 

The National Council of Teachers of Mathematics has advanced its 
recommendations for mathematics education in three volumes referred to 
collectively in this paper as the NCTM Standards. The first volume, the 
Curriculum and Evaluation Standards for School Mathematics (1989), outlines 
what the NCTM believes is important for students to learn and presents a 
vision for how students should learn. The evaluation standards focus on 
what to assess and how to assess it. The Professional Standards for Teaching 
Mathematics (1991) expands on that vision of how students should be 
taught and prescribes principles for the development and evaluation of 
teachers. The Assessment Standards for School Mathematics (1995) expands on 
the evaluation standards from the first volume. 

The theme of the NCTM Standards is that students should learn to think 
mathematically. This theme is reflected in the earliest philosophical 
underpinnings of the ACT Assessment Mathematics Test, a test designed 
to measure what students can do with what they know — their 
mathematical reasoning — rather than concentrating on recall of formulas, 
knowledge of facts, or definitions of terms. 

All of the topics recommended in the NCTM Standards appear in the 
content specifications for the ACT Assessment Mathematics Test. Over the 
last few years, many of the topics that the NCTM Standards recommend be 
deemphasized have accordingly decreased in frequency on the ACT 
Assessment Mathematics Test, and topics that the NCTM Standards 
recommend receive increased emphasis have grown in frequency. 

However, the ACT Assessment Mathematics Test does not extend to the 
full depth of the NCTM Standards -recommended curriculum. For example, 
the probability questions on the ACT Assessment Mathematics Test 
currently focus on more basic concepts of probability, while the NCTM 
Standards recommend a broader range of probability topics. As the 
curriculum continues to grow and students master more advanced 
probability concepts, the ACT Assessment Mathematics Test will measure 
those skills. 

The NCTM Standards discuss problem solving, mathematical 
communication, reasoning, and making mathematical connections. The 
ACT Assessment Mathematics Test addresses all of these key areas. In 
problem solving, the test strives for true-to-life contexts that the students 
will see as important mathematically. The ACT Assessment Mathematics 
Test measures communication skills largely in the areas of mathematical 
reading skills and recognizing appropriate written mathematics. 
Mathematical reasoning is the foundation of the ACT Assessment 
Mathematics Test: there is significant agreement between the NCTM 
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Standards and the ACT Assessment Mathematics Test in this area. As to 
making mathematical connections, many problems on the ACT Assessment 
Mathematics Test cross mathematics-topic boundaries; some of these 
questions require an integrated knowledge of several topics, or connecting 
mathematics with the everyday world. In addition to these goals, the 
NCTM Standards call for integrating mathematics with other content areas. 
The ACT Assessment Mathematics Test presents its problems in real-world 
contexts; however, to avoid confusion over what is being measured, the 
test does not require specific knowledge in these other content areas in 
order to be able to answer the questions correctly. 

ACT policy emphasizes the need for multiple sources of assessment 
information for making decisions about students. Similarly, the NCTM 
Standards also call for the use of multiple types of assessments: multiple- 
choice, short-answer, discussion, or open-ended questions; structured or 
open-ended interviews; homework; projects; journals; essays; 
dramatizations; and class presentations. Both organizations have a 
common goal — that of basing decisions about students on as much 
information as possible about what they know and can do. 

The NCTM Standards recommend that calculators and computers be 
available to students at all times. Within limits of test security, students 
may bring whatever calculator they wish to use for the ACT Assessment 
Mathematics Test. All questions, however, can be answered without a 
calculator. 

The goal of ACT, and of the NCTM Standards, is to help students be 
successful in mathematics throughout and after high school. Mathematical 
reasoning is the common vehicle. Because the ACT Assessment 
Mathematics Test is designed to measure mathematics skills that are 
important for success in college, and many of these skills are also reflected 
in the NCTM Standards, they share a common goal and theme. 

Textbook Review 

To survey what kinds of materials teachers are using in the high school 
classroom, ACT reviews a wide variety of mathematics textbooks. There 
are currently three types of high school mathematics textbooks — 
traditional, integrated, and NCTM Standards -based. 

The content and presentation of the traditional textbook series reflect a 
lecture-based approach, accompanied by a set of problems focusing on a 
particular skill, with some features added to meet NCTM Standards. Some 
of these features are geometric modeling to clarify algebraic concepts and 
suggested activities to use a calculator or computer; most are additions to 
the existing approach. The practice exercises contain sets of problems that 
are very similar in nature, some of which may include application 
problems. (A variation on the "traditional" type contains traditional 
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content but is organized by problem sets intended to promote continual 
practice.) 

In the early 1990s there was a push to fully integrate two years of algebra 
with one of geometry. Several new textbook series were then published. 
These integrated series have not yet been widely accepted even though 
several states now require such integration. Teachers have indicated that 
time and training are needed to make these textbook changes. 

The most recent development has been to incorporate the NCTM Standards 
into mathematics textbooks. These new series expand and reorganize the 
mathematical content while integrating technology. Influenced by the 
NCTM Standards, the content and presentation in these textbooks are 
notably different from those of traditional series. More attention is given to 
real-world applications and to the understanding of underlying 
mathematical principles. The content integrates data interpretation, 
probability, statistics, and discrete mathematics topics into all mathematics 
courses. The content is also introduced in a new order of presentation. For 
example, functions are presented in the first year of algebra instead of the 
second; and matrices appear in some first-year algebra textbooks in the 
NCTM Standards -based series. 

A wide variety of textbooks appear in high school mathematics classrooms. 
ACT uses textbooks to corroborate survey evidence about the timing of the 
introduction of mathematics topics to students. The curriculum is not 
static, however — both the NCTM Standards and evolving technologies are 
greatly influencing the mathematics taught and the way it is taught. ACT 
continually monitors these developments and takes them into account in 
developing the ACT Assessment Mathematics Test. 

INVOLVEMENT OF U.S. MATHEMATICS TEACHERS IN THE 

Test Development Process 

Production of Test Questions 

ACT has three goals in selecting question writers to develop its tests: 

• to work with classroom teachers who have day-to-day knowledge of 
what is happening in mathematics classrooms nationwide; 

• to achieve balanced perspectives of high school teachers and college 
faculty; and 

• to ensure that the question writers represent the diversity of the 
population of the United States with respect to ethnic background, 
gender, and geographic location. 
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ACT recruits question writers who represent a cross section of educators 
within the United States and to reflect the diversity of the population 
served by the ACT Assessment. Writers are trained in sound question 
writing practices by ACT staff, and they work closely with ACT test 
specialists, who assist them in producing questions of high quality that 
meet the test specifications. 

Review of Test Questions 

High school and college mathematics teachers are also directly involved in 
the process of ACT Assessment test-question review. They are involved in 
two kinds of review: fairness and content accuracy. Along with other 
persons sensitive to issues of test and question fairness, teachers are a part 
of the fairness review process. They examine all prospective test questions 
and passages for ethnic and gender fairness and representativeness. 

High school and college mathematics teachers participate in verifying the 
content accuracy of the ACT Assessment as well. They ensure, by 
reviewing all prospective test questions, that the test materials are accurate 
and representative of the curriculum, as well as grade appropriate. 

A minimum of 20 independent reviews are performed on each test 
question during the 2Vi years before it appears on a national ACT 
Assessment test. The many reviews help ensure that each student's level of 
achievement is accurately and fairly measured. The teachers' perspective is 
vital in every stage of the test development process. 

ACT Research 



Course Placement Research 

If the purpose of the ACT Assessment is to measure that which is taught 
in high school that is important for college success, then it follows that the 
relationship of test performance to college grade point average is a valid 
criterion of the ACT Assessment's success. ACT conducts considerable 
research into this relationship and other associated topics to ensure that 
the tests measure what ACT says they measure for the purposes for which 
the tests are being used (validity) and that the tests will be highly accurate 
in their measurement (reliability). Research on predictive validity, for 
example, examines how well the ACT Assessment can predict college 
success. Through its Course Placement Service, ACT works with individual 
colleges and universities to establish the relationship between the ACT 
Assessment scores and high school grade point average and college 
success, based on students' performance in the courses in which they are 
placed, act's purpose is to demonstrate that the ACT Assessment predicts 
college success accurately and effectively. 
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Colleges use ACT Assessment scores alone or in combination with other 
factors in making course placement decisions. Among participants in 
act's Course Placement Service, about 92% have used the ACT 
Assessment to help decide whether students should enroll in a standard 
course or in some kind of remedial course in areas of language arts, 
science, and mathematics. About 8% of the participating schools have used 
ACT Assessment scores to decide between a standard course and some 
kind of advanced course. 

Mathematics Test scores are the most commonly used of the single-test 
scores in placement decisions. (Mathematics Test scores are often also used 
in combination with Science Reasoning scores.) In making placement 
decisions. Mathematics Test scores are most often used to assign students 
to college mathematics courses at the following levels: pre-algebra, 
intermediate algebra, college algebra, and calculus. ACT's research shows 
that placement in mathematics has a high level of success: on average at 
least 65% of students are accurately placed based on their Mathematics 
Test score, either alone or with other factors. Results of this same research 
indicate that the median accurate placement rate in intermediate algebra, 
for example, was approximately 69%. If no Mathematics Test score had 
been used in making the decision and all students had been permitted to 
take the course, the expected accuracy rate would have been 41%. This is 
an increase in accuracy of 28% over what would have been expected 
without using the ACT Assessment Mathematics Test score. Table 5 
presents the data for other mathematics courses. 

Table 5 

ACT Assessment Mathematics Test Cutoff Scores 
and Decision-Based Validity Statistics 
for Course Placement 



Course Type 


No. of 
Institutions 


Median 

Cutoff 

Score 


Median 

Accuracy 

Rate 


Median 
Increase in 
Accuracy 
Rate 


Intermediate Algebra 


15 


22 


69 


28 


College Algebra 


44 


23 


71 


28 


Statistics/Probability 


4 


23 


77 


35 


Pre-Calculus 


14 


25 


72 


36 


Trigonometry 


16 


26 


71 


33 


Calculus 


34 


26 


78 


47 



Such increases in accuracy for course placement support the usefulness of 
the ACT Assessment Mathematics Test for college placement. This is a 
direct result of the close relationship between the ACT Assessment 
Mathematics Test and the mathematics curriculum nationally. 
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Course Work Research 

An additional area of validity research addresses the impact of course 
work on test scores. ACT conducts considerable validity research that 
seeks confirmation that the ACT Assessment is a sound measure of 
educational development. Results of this research repeatedly confirm that 
students who meet core course requirements (4 years of English, and 3 
years each of mathematics, social studies, and science) will perform better 
on the ACT Assessment than those who do not. ACT's research has clearly 
demonstrated that students taking or planning to take more mathematics 
courses perform better than those who take or plan to take fewer. 
Moreover, those students who take more mathematics courses in high 
school score higher than those who take fewer (1-2) mathematics courses. 

In a 1992 study, ACT researchers used a representative sample of 40,000 
ACT Assessment -tested 1989-90 high school juniors and seniors from 400 
high schools. The researchers found that, for each additional mathematics 
course taken, students' Mathematics Test scale scores increased, on 
average, by 1.27 scale score points. This finding — that students who have 
taken and/or plan to take more mathematics courses (and in the fall of 
grade 12 students can plan for at most only one semester of high school 
courses) achieve higher Mathematics Test scores than those who do not — 
confirms the close relationship between the ACT Assessment Mathematics 
Test and the mathematics curriculum at the high school level, and further 
documents the curriculum-based nature of the test. 

Summary and Conclusions 



The content of the ACT Assessment Mathematics Test is based on a 
complex analysis of the high school curricula and what colleges require for 
success in entry-level and advanced-level courses. Results of ACT's studies 
confirm that the ACT Assessment Mathematics Test is firmly grounded in 
the curriculum and measures content and skills students learn in high 
school that are important for college success. 

ACT's national curriculum surveys, reviews of state curriculum 
frameworks, and monitoring of national standards form the basis for the 
knowledge/skill continuum underlying the test specifications. Mathematics 
educators, notably high school teachers, curriculum specialists, and college 
faculty, all play important roles in all phases of ACT Assessment 
Mathematics Test development. Finally, ACT research provides strong 
evidence that the test predicts college success and is useful for accurate 
placement of students in college courses. 

The relationship between the ACT Assessment Mathematics Test and the 
high school and college-entry mathematics curriculum is a close one. 
Because the curriculum is dynamic, ACT continuously monitors national 
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developments related to curriculum standards, changes in state curriculum 
frameworks, new developments in technology and textbooks, and other 
factors that affect the curriculum taught in the schools. ACT also continues 
to explore new assessment alternatives that offer students additional ways 
of demonstrating what they know and can do in mathematics. Finally, and 
most importantly, ACT continues to rely on the critical contributions that 
mathematics educators at the secondary and postsecondary levels make to 
all phases of the test development process. 
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